NONLINEAR INTEGER OPTIMIZATION AND APPLICATIONS IN BIOMEDICINE By OLEG A. PROKOPYEV A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA

of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy NONLINEAR INTEGER OPTIMIZATION AND APPLICATIONS IN BIOMEDICINE By Oleg A. Prokopyev August 2006 Chair: Panos M. Pardalos Major Department: Industrial and Systems Engineering In this dissertation we consider fractional and quadratic 0-1 optimization problems with some related applications in biomedicine. First, we discuss fractional 0–1 programming problems. New results on computational complexity of various classes of fractional 0–1 programming problems, equivalent reformulations as well as some heuristic approaches are reported. In part, this research was motivated by a new fractional 0–1 programming model for biclustering, an important data mining problem, which has a great significance for biomedical applications. In the second part of the dissertation we investigate quadratic 0–1 optimization problems. We are mostly concerned with two important applications of quadratic 0–1 programming: in silico sequence selection in de novo protein design and epileptic seizure prediction. In the first application, we focus on the mathematical formulations for in silico sequence selection in de novo protein design. We discuss linear mixed 0–1